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DETAILED ACTION 

1. This communication is responsive to Amendment A, filed 12/03/2003. 

2. Claims 1-38 are pending in this application. Claims 1, 9, 13, 21, 23, 29, 31, 34, 35, 38 
are independent claims. In the Amendment A, claims 20, 29, 34, 35, 38 have been amended. 
This action is made Final. 

3. The objection to the specification (claim objection) of the invention has been withdrawn 
in view of the amendment. 

Claim Rejections - 35 USC § 102 

4. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the 
basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless: 
(e) the invention was described in 

(1) an application for patent, published under section 122(b), by another filed in the United States before 
the invention by the applicant for patent or 

(2) a patent granted on an application for patent by another filed in the United States before the invention 
by the applicant for patent, except that an international apphcation filed under the treaty defined in section 
35 1(a) shall have the effects for purposes of this subsection of an apphcation filed in the United States only 
if the international application designated the United States and was published under Article 21(2) of such 
treaty in the English language. 

5. Claims 21-22 are rejected under 35 U.S.C. 102(e) as being anticipated by Van Zoest et al. 
(US Patent No. 6,496,802 Bl). 

Van Zoest anticipated independent claim 21, by the following: 
As per claim 21, Van Zoest teaches "a timestamp module for assigning timestamps to 
audio samples that are to be rendered by a media player renderer" at col. 4, lines 57-65; 
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"a spectrum analyzer for processing the audio samples to provide frequency data 
associated with the audio samples" at col. 16, lines 9-43; 

"muhiple data structures each of which being associated with an audio sample, the data 
structures each containing timestamp data and frequency data for its associated audio sample" at 
col. 5, lines 8-19, col. 6, lines 18-25, col. 9, lines 24-65, col. 4, lines 57-65, col. 16, lines 9-14, 
Fig. 2; 

"the system being configured to use the timestamp data to ascertain a data structure 
associated with an audio sample that is currently being rendered by the media player renderer 
and provide the frequency data associated with that audio sample so that the frequency data can 
be used to render a visualization associated with that audio sample" at col. 5, lines 8-19, col. 16, 
lines 9-15, col. 18, lines 55-61. 

As per claim 22, Van Zoest teaches "the spectrum analyzer comprises a Fast Fourier 
Transform that is utilized to provide the frequency data" at col. 16, lines 9-14. 

Claim Rejections - 35 USC §103 
6. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

This application currently names joint inventors. In considering patentability of the 
claims under 35 U.S.C. 103(a), the examiner presumes that the subject matter of the various 
claims was commonly owned at the time any inventions covered therein were made absent any 
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evidence to the contrary. Applicant is advised of the obHgation under 37 CFR 1.56 to point out 
the inventor and invention dates of each claim that was not commonly owned at the time a later 
invention was made in order for the examiner to consider the applicability of 35 U.S.C. 103(c) 
and potential 35 U.S.C. 102(e), (f) or (g) prior art under 35 U.S.C. 103(a). 

7. Claims 35-38 are rejected under 35 U.S.C. 103(a) as being unpatentable over Prasad et al. 
(US Patent No. 6,269,122 Bl), in view of Milne et al. (US Patent No. 5,655,144). 
Prasad anticipated independent claims 35, 38, by the following: 

As per claim 35, Prasad teaches "defining a frame rate at which visualization frames are 
to be rendered" at col. 6, lines 1-60; 

"setting a threshold associated with an amount of time that is to be spent rendering a 
visualization frame" at col. 6, lines 1-60; 

"monitoring the time associated with rendering individual visualization frames" at col. 6, 
lines 1-60, col. 7, Hnes 29-59; 

"determining whether a visualization frame rendering time exceeds the threshold" at col. 
6, lines 1-60, col. 7, lines 1-28; 

"and providing an effective frame rate for rendering visualization frames that is longer 
than the defined frame rate if the determined visualization frame rendering time exceeds the 
threshold" at col. 6, lines 1-60, col. 7, lines 29-59. 

Prasad does not explicitly teach "the visualization fi-ames rendered fi-om characterizing 
data that is computed from audio sample and which is used to create the visualization". 
However, Milne teaches this limitation at col. 17, lines 15-39. 

Thus, it would have been obvious to one of ordinary skill in the art at the time of the 
invention to modify the teachings of Prasad with the teachings of Milne to include "the 
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visualization frames rendered from characterizing data that is computed from audio sample and 
which is used to create the visualization" in order to display the visual clock object being 
synchronized with the audio clock object as taught by Milne at col. 19, lines 8-11. 

As per claim 38, Prasad teaches "set a threshold associated with an amount of time that 
is to be spent rendering a visualization frame for a given frame rate" at col. 6, lines 1-60; 

"monitor the time associated with rendering individual visualization frames (step of 
determining the total elapsed time)" at col. 6, lines 1-60, col. 7, lines 1-28; 

"determine whether a visualization frame rendering time exceeds the threshold" at col. 6, 
lines 1-60, col. 7, lines 1-28; 

"provide an effective frame rate (step of adjusting the rate of playback of the recorded 
video stream 41) for rendering the visualization that is longer than the defined frame rate if the 
determined visualization frame rendering time exceeds the threshold" at col. 5, line 48 to col. 6, 
line 60. 

Prasad does not specifically teach "said visualization frame being associated with a 
visualization that is rendered using characterizing data that is computed from audio samples, 
which characterizing data is used to create the visualization". However, Milne teaches this 
limitation at col. 17, lines 15-39. 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
to modify the teachings of Prasad with the teachings of Milne to include "said visualization 
frame being associated with a visualization that is rendered using characterizing data that is 
computed from audio samples, which characterizing data is used to create the visualization" in 
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order to display the visual clock object being synchronized with the audio clock object, as taught 
by Milne at col. 19, lines 8-1 L 

As per claim 36, Prasad teaches "increasing a call interval associated with calls that are 
made to a visualization-rendering component" at col. 5, lines 48-67. 

As per claim 37, Prasad teaches "modifying the effective frame rate so that the 
visualization frames are rendered at the defined frame rate" at col. 6, line 61 to col. 7, lines 59. 

8. Claims 1-6, 9-11, 13, 16, 17, 23-26, 31 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Milne et al. (US Patent No. 5,655,144), in view of Jang et al. (US Patent No. 
6,442,758 61). 

As per claim 1, Milne teaches "one or more audio sources configured to provide audio 
samples that are to be rendered by a media player" at col. 16, lines 13-19, col. 15, lines 48-60; 

"one or more effects configured to receive the characterizing data (clock rate) and use the 
characterizing data to render a visualization that is synchronized with an audio sample that is 
being rendered by the media player" at col. 16, lines 63-64, col. 19, lines 1-11, col. 15, lines 39- 
59, col. 18, lines 1-12, and Figs 11,37. 

Milne teaches "an audio sample pre-processor communicatively linked with the one or 
more audio sources and configured to receive and pre-process audio samples before the samples 
are rendered, the pre-processing providing characterizing data associated with each sample" at 
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col. 19, lines 1-11, col. 16, lines 62-64, col. 15, lines 39-59, and Fig. 25. However, Milne does 
not expressly teach a pre-processor. 

Jang teaches the pre-processor 1 17 at col. 11, lines 1-2, Figure 6, 
Thus, it would have been obvious to one of ordinary skill in the art at the time of the 
invention to modify the teachings of Milne with the teachings of Jang to include "an audio 
sample pre-processor communicatively linked with the one or more audio sources and 
configured to receive and pre-process audio samples before the samples are rendered, the 
pre-processing providing characterizing data associated with each sample" in order to split the 
audio signal into stereo signals. 

As per claim 9, Milne teaches "one or more effects configured to receive the frequency 
data and use the frequency data to render a visualization that is synchronized with an audio 
sample that is being rendered by the media player" at col. 16, lines 63-64, col. 19, Hnes 1-11, col. 
15, lines 39-59, col. 18, lines 1-12, and Figs 11, 37. 

Milne teaches "an audio sample pre-processor configured to receive and pre-process 
audio samples before the samples are rendered by the media player, the pre-processing providing 
frequency data associated with each sample" at col 16, lines 62-64, col. 15, lines 39-59, and Fig. 
25. Milne, however, does not specifically teach a pre-processor, 

Jang teaches the pre-processor 1 17 at col. 11, lines 1-2, Figure 6. 

Thus, it would have been obvious to one of ordinary skill in the art at the time of the 
invention to modify the teachings of Milne with the teachings of Jang to include "an audio 
sample pre-processor configured to receive and pre-process audio samples before the samples are 
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rendered by the media player, the pre-processing providing frequency data associated with each 
sample" in order to achieve greater bandwidth. 

As per claim 13, Milne teaches "multiple data structures configured to hold the 
characterizing data, each to data structure being associated with an audio sample" at col, 17, lines 
13-58, col. 4, lines 4-17; 

"an audio rendering object configured to call the audio sample pre-processor to ascertain 
the characterizing data associated with an audio sample that is currently being rendered by the 
renderer" at col. 16, lines 21-39, col. 6, lines 21-33, col. 8, lines 32-39; 

"one or more effects configured to receive characterizing data that is associated with the 
data structure (graphic objects related by time) having the timestamp that is nearest in value to 
said time, and use the characterizing data to render a visualization that is synchronized with the 
audio sample that is being rendered by the renderer" at col. 16, lines 22-39, col. 19, lines 1-11, 
col 17, lines 15-63, Fig. 12, Fig. 36. 

"the audio sample pre-processor being configured to ascertain said characterizing data by 
querying the renderer for a time associated with the currently-rendered audio sample, and then 
using said time to identify a data structure having a timestamp that is nearest in value to said 
time" at col. 16, lines 22-39, col. 19, lines 1-1 1, col. 17, lines 15-63, col. 9, lines 43-61, col. 8, 
lines 40-65; 

"an audio sample pre-processor configured to receive and preprocess audio samples 
before the samples are rendered by a renderer that comprises part of a media player (col. 16, 
lines 62-64, col. 15, lines 39-59, Fig. 25), the audio sample preprocessor preprocessing the 
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samples to provide characterizing data associated with each sample, the characterizing data 
comprising a timestamp associated with each audio sample, the timestamp being assigned in 
accordance with when the audio sample is calculated to be rendered by the renderer" at col. 19, 
lines 1-12, col. 16, lines 13-39, col 15, lines 48-60, col. 16, line 40 to col. 17, line 12. 

However, Milne does not specifically teach a pre-processor. 

Jang teaches the pre-processor 1 17 at col. 1 1, lines 1-2, Figure 6. 

Thus, it would have been obvious to one of ordinary skill in the art at the time of the 
invention to modify the teachings of Milne with the teachings of Jang to include "a pre- 
processor" in order to split the audio signal into stereo signals being sent to a respective audio 
bus within the audio bus system. 

As per claim 23, Milne teaches "receiving multiple audio samples" at col 16, Hnes 13- 
39, col 15, lines 48-60; 

"determining when an audio sample is being rendered by the media player renderer" at 
col 19, lines 1-1 1, col 17, lines 15-63, col 16, lines 21-39; 

"responsive to said determining, using the characterizing data that is associated with the 
audio sample that is being rendered to provide a visualization" at col. 16, lines 21-39, col. 19, 
lines 1-11, col 17, lines 13-58; 

"pre-processing the audio samples before they are rendered by a media player renderer, 
the pre-processing providing characterizing data for each sample" at col 16, lines 13-39, col 15, 
lines 48-60. 

However, Milne does not explicitly teach a pre-processor. 
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Jang teaches the pre-processor 1 17 at col. 1 1, lines 1-2, Figure 6. 

Thus, it would have been obvious to one of ordinary skill in the art at the time of the 
invention to modify the teachings of Milne with the teachings of Jang to include "pre-processing 
the audio samples before they are rendered by a media player renderer, the pre-processing 
providing characterizing data for each sample" in order to split the audio signal into stereo 
signals being sent to a respective audio bus within the audio bus system. 

As per claim 31, Milne teaches "calling the media player renderer for a time associated 
with a currently rendered audio sample" at col 16, lines 21-39, col. 19, lines 1-11; 

"using the time to select a data structure containing characterizing data associated with 
the currently-rendered audio sample" at col. 17, lines 13-58, col. 19, lines 1-12, Fig. 37; 

"providing the characterizing data to a component for rendering a visualization" at col. 
19, lines 1-19, Fig. 37, col. 17, lines 1-12, col. 17, lines 15-63. 

"calling an audio sample pre-processor for characterizing data that is associated with an 
audio sample that is currently being rendered by a media player renderer" at col. 16, lines 13-39, 
col. 15, lines 48-60. 

However, Milne does not expressly teach a pre-processor. 

Jang teaches the pre-processor 1 17 at col. 1 1, lines 1-2, Figure 6, 

Thus, it would have been obvious to one of ordinary skill in the art at the time of the 
invention to modify the teachings of Milne with the teachings of Jang to include "calling an 
audio sample pre-processor for characterizing data that is associated with an audio sample that is 
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currently being rendered by a media player renderer" in order to split the audio signal into stereo 
signals being sent to a respective audio bus within the audio bus system. 

As per claim 2, Milne teaches "multiple data structures configured to hold the 
characterizing data, each data structure being associated with an audio sample" at col. 4, lines 
4-17. 

As per claim 3, Milne teaches "the audio sample pre-processor is configured to maintain 
the data structures" at col. 8, line 57 to col 9, line 13. 

As per claim 4, Milne teaches "the audio sample pre-processor comprises a timestamp 
module that provides a timestamp for each audio sample, each timestamp being maintained by a 
data structure associated with the audio sample" at col 16, line 13 to col 17, line 12. 

As per claim 5, Milne teaches "the timestamp is assigned by the timestamp module 
based upon when the audio sample is calculated to be rendered by the media player" at col 16, 
line 13 to col. 17, line 12. 

As per claim 6, Milne teaches "the audio sample pre-processor is configured to: query a 
media player audio sample renderer for a time associated with an audio sample that is being 
currently rendered" at col. 15, line 48 to col. 16, line 39, 

"and use the time to ascertain a timestamp of an associated audio sample, the to audio 
sample pre-processor further being configured to provide characterizing data of the associated 
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audio sample so that the characterizing data can be used to render the visualization" at col. 17, 
lines 14-63, col. 19, lines 1-12, Fig. 36. 

As per claim 10, Milne teaches "multiple data structures configured to hold the 
frequency data, each data structure being associated with an audio sample" at col. 4, lines 4-17. 

As per claim 11, Milne teaches "query a media player audio sample Tenderer for a time 
associated with an audio sample that is being currently rendered" at col. 15, line 48 to col. 16, 
line 39, 

"and use the time to ascertain a timestamp of an associated audio sample, the audio 
sample pre-processor further being configured to provide frequency data of the associated audio 
sample to the one or more effects so that the frequency data is can be used to render the 
visualization" at col. 17, lines 14-63, col. 19, lines 1-12, Fig. 36. 

As per claim 16, Milne teaches "the visualization is rendered in a rendering area in 
which other media types can be rendered" at col, 17, lines 15-63, Figs. 12, 36. 

As per claim 17, Milne teaches "the other media types comprise a video type" at col. 17, 
lines 15-63, Figs. 12, 36. 

As per claim 24, Milne teaches "maintaining characterizing data for each audio sample 
in a data structure associated with each audio sample" at col. 4, lines 4-17. 
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As per claim 25, Milne teaches "the characterizing data comprises a timestamp 
associated with the audio sample, the timestamp being provided based upon when the audio 
sample is calculated to be rendered by the media player renderer" at coL 16, line 40 to col. 17, 
line 12. 

As per claim 26, Milne teaches "ascertaining a time associated with a currently-rendered 
audio sample" at col. 19, lines 1-12; 

"selecting a data structure having a timestamp that is nearest the time" at col 19, lines 1- 

12; 

"providing characterizing data associated with the selected data structure to a component 
configured to provide the visualization" at col. 17, lines 1-12, col. 17, lines 15-63. 

9. Claims 7-8, 12, 14-15, 27-29, 30, 32-34 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Milne et al. (US Patent No. 5,655,144) and Jang et al. (US Patent No. 
6,442,758 Bl), and further in view of Van Zoest et al. (US Patent No. 6,496,802 Bl). 

As per claim 29, Milne teaches "receiving multiple audio samples" at col. 15, line 61 to 
col. 16, line 20; 

"maintaining frequency data arid a timestamp for each sample in a data structure" at col. 
16, lines 21-59; 

"determining when an audio sample is being rendered by the media player renderer" at 
col. 16, lines 21-39; 

"ascertaining a time associated with a currently-rendered sample at col. 16 lines 21-39; 
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"selecting a data structure (graphic objects related by time ) having a timestamp that is 
nearest the time" at col. 17, lines 13-58; 

"providing frequency data (clock rate) associated with the selected data structure (A 
graphic sequence provides a member function that maps a time within the duration of the 
sequence to a particular graphic object) to a component configured to use the frequency data to 
render the visualization" at col. 17, lines 15-27, col. 19, lines 1-11, col. 16, lines 40-59, Fig. 37. 

Milne does not specifically teach a pre-processor and a Fast Fourier Transform. 
Jang teaches the audio pre-processor 1 17 in Figure 6. 

It thus would have been obvious to one of ordinary skill in the art at the time of 
the invention to modify the teachings of Milne with the teachings of Jang to include a 
"pre-processor" in order to provide a multimedia system having a time base correction 
function between the incoming audio sample rate and that which is derived from the 
system master clock, as taught by Jang at col. 13, lines 38-41. 

Furthermore, Van Zoest teaches the Fast Fourier Transform at col. 16, lines 9-15. 

It would have been obvious to one of ordinary skill in the art at the time of the 
invention to modify the teachings of Milne, Jang with the teachings of Van Zoest to 
include "a Fast Fourier Transform" in order to provide a multimedia system having a 
frequency analysis of data as taught by Van Zoest at col. 16 line 17. By doing this, the 
system could verify and identifying the corresponding portions of the samples with 
maximum correlation, as taught by Van Zoest at col. 16, lines 4-6. 
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As per claim 34, Milne teaches "query for frequency data that is associated with an audio 
sample that is currently being rendered by the media player renderer" at col. 16, lines 16-59. 
"query for a time associated with the currently-rendered audio sample" at col. 16, lines 

16-59; 

"use the time to select a data structure containing frequency data associated with the 
currently-rendered audio sample" at col 17, Unes 15-27, col. 19, lines 1-11, col. 16, lines 40-59 
and Fig. 37. 

"provide the frequency data to a component for rendering a visualization" at col. 17, 
lines 15-27, col. 19, lines 1-11, col. 16, lines 40-59 and Fig. 37. 

Milne teaches the pre-processing audio sample, which is performed by a host processor 
as shown in Fig. 25. But Milne does not specifically disclose the term "pre-processor". 

Milne does not specifically teach a pre-processor and a Fast Fourier Transform. 
Jang teaches the audio pre-processor 1 17 in Figure 6. 

It thus would have been obvious to one of ordinary skill in the art at the time of 
the invention to modify the teachings of Milne with the teachings of Jang to include a 
"pre-processor", in order to provide a muhimedia system having a time base correction 
function between the incoming audio sample rate and that which is derived from the 
system master clock, as taught by Jang at col. 13, lines 38-41. 

In addition. Van Zoest teaches the Fast Fourier Transform at col. 16, lines 9-15. 

It would have been obvious to one of ordinary skill in the art at the time of the 
invention to modify the teachings of Milne, Jang with the teachings of Van Zoest to 
include "a Fast Fourier Transform" in order to provide a muhimedia system having a 
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frequency analysis of data as taught by Zoest at col. 16 line 17. By doing this, the system 
could verify and identifying the corresponding portions of the samples with maximum 
correlation, as taught by Van Zoest at col. 16, lines 4-6. 

As per claims 7, 14, Milne and Jang do not expressly teach "characterizing data 
comprises frequency data". However, Van Zoest teaches this limitation at col 16, lines 9-14. 

Thus, it would have been obvious to one of ordinary skill in the art at the time of the 
invention to modify the teachings of Milne, Jang with the teachings of Van Zoest to include 
"characterizing data comprises frequency data" in order to provide a system and method for 
providing access to electronic works to users over a network. 

As per claim 8, Milne and Jang do not explicitly teach "a Fast Fourier Transform that it 
utilizes to process the audio samples to provide frequency data associated with the audio 
samples". However, Van Zoest teaches this limitation at col. 16, lines 9-14. 

Thus, it would have been obvious to one of ordinary skill in the art at the time of the 
invention to modify the teachings of Milne, Jang with the teachings of Van Zoest to include "a 
Fast Fourier Transform that it utilizes to process the audio samples to provide frequency data 
associated with the audio samples" in order to provide a system and method for providing access 
to electronic works to users over a network. 
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As per claim 12, Milne and Jang do not specifically teach "the audio sample 
pre-processor pre processes the audio samples by using a Fast Fourier Transform to provide the 
frequency data". However, Van Zoest teaches this limitation at col. 16, lines 9-14, 

Thus, it would have been obvious to one of ordinary skill in the art at the time of the 
invention to modify the teachings of Milne, Jang with the teachings of Van Zoest to include "the 
audio sample pre-processor pre processes the audio samples by using a Fast Fourier Transform to 
provide the frequency data" in order to provide a system and method for providing access to 
electronic works to users over a network. 

As per claims 15, 28, Milne and Jang do not explicitly teach "the audio sample 
pre-processor comprises a Fast Fourier Transform that it utilizes to process the audio samples to 
provide frequency data associated with the audio samples". However, Van Zoest teaches this 
limitation at col. 16, lines 9-14. 

Thus, it would have been obvious to one of ordinary skill in the art at the time of the 
invention to modify the teachings of Milne, Jang with the teachings of Van Zoest to include "the 
audio sample pre-processor comprises a Fast Fourier Transform that it utilizes to process the 
audio samples to provide frequency data associated with the audio samples" in order to provide a 
system and method for providing access to electronic works to users over a network. 

As per claims 27, 32, Milne and Jang do not specifically teach "the characterizing data 
comprises fi*equency data associated with each sample". However, Van Zoest teaches this 
limitation at col. 16, lines 9-14. 
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Thus, it would have been obvious to one of ordinary skill in the art at the time of the 
invention to modify the teachings of Milne, Jang with the teachings of Van Zoest to include "the 
characterizing data comprises frequency data associated with each sample" in order to provide a 
system and method for providing access to electronic works to users over a network. 

As per claim 30, Milne teaches "One or more computer-readable media having 
computer-readable to instructions thereon which, when executed by a computer, cause the 
computer to implement the method of claim 29" at col. 3, line 51 to col. 4, line 46. 

As per claim 33, Milne and Jang do not specifically teach "the characterizing data 
comprises frequency data associated with the audio samples and generated by pre-processing the 
audio samples using a Fast Fourier Transform". However, Van Zoest teaches this limitation at 
col. 16, lines 9-14. 

Thus, it would have been obvious to one of ordinary skill in the art at the time of the 
invention to modify the teachings of Milne, Jang with the teachings of Van Zoest to include "the 
characterizing data comprises frequency data associated with the audio samples and generated by 
pre-processing the audio samples using a Fast Fourier Transform" in order to provide a system 
and method for providing access to electronic works to users over a network. 

12. Claims 18-20 are rejected under 35 U.S.C. 103(a) as being unpatentable over Milne et al. 
(US Patent No. 5,655,144) and Jang et al. (US Patent No. 6,442,758 Bl), and further in view of 
Chemock et al. (US Patent No. 6,3 14,569 Bl). 



Application/Control Number: 09/8 1 7,902 Page 1 9 

Art Unit: 2177 

As per claim 18, Milne and Jang do not specifically teach "the other media types 
comprise a skin type". However, Chemock teaches this limitation at col. 6, lines 36-44. 

Thus, it would have been obvious to one of ordinary skill in the art at the time of the 
invention to modify the teachings of Milne, Jang with the teachings of Chemock to include "the 
other media types comprise a skin type" in order to provide a system which enables the display 
or playing of audio, video, or graphics objects in tandem with the video and audio play of a 
digital video presentation. 

As per claim 19, Milne and Jang do not specifically teach "the other media types 
comprise a HTML type". However, Chernock teaches this limitation at col. 7, lines 1-8. 

Thus, it would have been obvious to one of ordinary skill in the art at the time of the 
invention to modify the teachings of Milne, Jang with the teachings of Chemock to include "the 
other media types comprise a HTML type" in order to provide a system which enables the 
display or playing of audio, video, or graphics objects in tandem with the video and audio play of 
a digital video presentation. 

As per claim 20, Milne and Jang do not specifically teach "the other media types 
comprise an animation type". However, Chernock teaches this limitation at col. 2, lines 36-48. 

Thus, it would have been obvious to one of ordinary skill in the art at the time of the 
invention to modify the teachings of Milne, Jang with the teachings of Chernock to include "the 
other media types comprise a animation type" in order to provide a system which enables the 
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display or playing of audio, video, or graphics objects in tandem with the video and audio play of 
a digital video presentation. 

Response to Arguments 

13. Applicant's arguments filed 12/03/2003 have been fully considered but they are not 
persuasive. 

Applicant argues that: 

(a) Milne and Jang references do not teach/suggest claim 1, the combination Milne and 
Jang appear to be based on hindsight reconstruction that has utilized Applicant's disclosure. 

(b) Milne and Jang references do not teach/suggest claim 9. 

(c) Milne and Jang references do not teach/suggest claim 13. 

(d) Milne and Jang references do not teach/suggest claim 23. 

(e) Milne and Jang references do not teach/suggest claim 31.. 

(f) Van Zoest reference does not teach/suggest claim 21. 

(g) Van Zoest, Jang, Chernock references do not teach/suggest claim 29. 

(h) Van Zoest reference does not teach/suggest claim 34. 

(i) Prasad reference does not teach/suggest claim 35. 
(j) Prasad reference does not teach/suggest claim 38. 

The Examiner respectfully disagrees for the following reasons: 

Per (a), with regards to claim 1, Milne teaches "one or more audio sources configured to 
provide audio samples that are to be rendered by a media player" at col. 16, lines 13-19, col. 15, 
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lines 48-60. More specifically, Milne discloses "the audio player can synchronize to an external 
clock. In this case, the audio player must insure that the sample being reproduced by the speaker 
corresponds to the current time of the external clock. For example, if at a given instant in time of 
the external clock object's current time is two seconds, then the 88200^ sample in the audio 
sequence should at that instant be reproduced by the speaker" (col. 16, lines 22-29). 

Milne teaches "one or more effects configured to receive the characterizing data and use 
the characterizing data to render a visualization that is synchronized with an audio sample that is 
being rendered by the media player" at col. 16, lines 63-64, col. 19, lines 1-11, col. 15, lines 39- 
59, col. 18, lines 1-12, and Figs 11,37. It should be noted the data characteristic corresponds to 
the "clock rate", and as shown in Fig 1 1, both audio and video sequences receive the 
characterizing data (i.e. clock rate) for synchronizing, and the characterizing data (i.e. clock rate) 
provided by the software clocks (col. 6, lines 63) is controlled by a microprocessor (col. 3, line 
58), and the synchronization between an audio object 3720 and a video object 3710 are shown in 
Fig. 37. 

Milne teaches the step of "an audio sample pre-processor communicatively linked with 
the one or more audio sources and configured to receive and pre-process audio samples before 
the samples are rendered, the pre-processing providing characterizing data associated with each 
sample" at col. 16, lines 62-64, col. 15, lines 39-59, and Fig. 25. More specifically, as shown in 
Fig. 25, the audio sample is provided characterizing data (i.e. clock rate) associated with each 
sample before the samples are rendered. It should be noted that the DAC and amplifier render 
the audio sample after the host processor (Fig. 25). 
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Although the function of the Milne's processor is as same as the function of the present 
invention's pre-processor which provides the frequency data (Specification, page 19, lines 4-5), 
Milne does not specifically use the term "pre-processor". However, Jang teaches the audio pre- 
processor 1 17 in Figure 6. Both Milne and Jang direct to the same field as the synchronization 
between the audio and video data based on the audio sample rate, it thus would have been 
obvious to one of ordinary skill in the art at the time of the invention to modify the teaching of 
Milne with the teaching of Jang to include "a pre-processor" in order to provide a multimedia 
system having a time base correction function between the incoming audio sample rate and that 
which is derived from the system master clock, as taught by Jang at col. 13, lines 38-41. 

In response to applicant's argument that the examiner's conclusion of obviousness is 
based upon improper hindsight reasoning, it must be recognized that any judgment on 
obviousness is in a sense necessarily a reconstruction based upon hindsight reasoning. But so 
long as it takes into account only knowledge which was within the level of ordinary skill at the 
time the claimed invention was made, and does not include knowledge gleaned only from the 
applicant's disclosure, such a reconstruction is proper. See In re McLaughlin, 443 F.2d 1392, 
170 USPQ 209 (CCPA 1971). In this case, the reconstruction is proper for the reasons submitted 
in the preceding paragraph. 

Furthermore, the Applicant's arguments seem to be suggesting that there is no suggestion 
to combine the references. In response to the arguments, the Examiner recognizes that 
obviousness can only be established by combining or modifying the teachings of the prior art to 
produce the claimed invention where there is some teaching, suggestion, or motivation to do so 
found either in the references themselves or in the knowledge generally available to one of 
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ordinary skill in the art. See In re Fine, 837 F.2d 1071, 5 USPQ2d 1596 (Fed. Cir. 1988) and In 
re Jones, 958 F.2d 347, 21 USPQ2d 1941 (Fed. Cir. 1992). In this case, the same reasoning by 
the Examiner is applicable to this argument can be found in the previous paragraph. 
Furthermore, as pointed out by the Examiner, only the teaching of preprocessing being taught by 
Jang is used in combining with the system of Milne to render obvious the claimed limitations. 

The Applicant has made a piecemeal analysis of the references. The Applicant is 
therefore reminded that one cannot show nonobviousness by attacking references individually 
where the rejections are based on combinations of references. See In re Keller, 642 F.2d 413, 
208 USPQ 871 (CCPA 1981); In re Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 
1986). 

Accordingly, the claimed invention as represented in the claims does not represent a 
patentable over the art of record. 

Per (b), with regards to claim 9, Milne teaches "one or more effects configured to receive 
the frequency data and use the frequency data to render a visualization that is synchronized with 
an audio sample that is being rendered by the media player" at col. 16, lines 63-64, col. 19, lines 
1-11, col. 15, lines 39-59, col. 18, lines 1-12, and Figs 11, 37. It should be noted the frequency 
data corresponds to the "clock rate", and as shown in Fig 1 1, both audio and video sequences 
receive the frequency data (i.e. clock rate) for synchronizing. Note that the characterizing data 
provided by the software clocks (col. 6, lines 63) is controlled by a microprocessor (col. 3, line 
58). 
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Milne teaches the step of "an audio sample pre-processor configured to receive and 
pre-process audio samples before the samples are rendered by the media player, the 
pre-processing providing frequency data associated with each sample" at col. 16, lines 62-64, 
col. 15, lines 39-59, and Fig. 25. More specifically, as shown in Fig. 25, the audio sample is 
provided frequency data (i.e. clock rate) associated with each sample before the samples are 
rendered. It should be noted that the amplifier (i.e. audio card) renders the audio sample after the 
host processor (Fig. 25). 

Milne teaches a software clock is responsible for providing the time base, while 
multimedia player resides in RAM, and under the control of CPU (col. 6, the last line). It should 
be noted that the time base is a frequency data (a time object is a floating point number that 
measure a point in time, measured in Pico seconds) (col. 7, line 24-34). Although Milne does not 
specifically teach the term "pre-processor", Jang discloses a pre-processor 1 17 as in Fig 6. 

Both Milne and Jang are directed to the same field as synchronizing video and audio data 
associated with the frequency data, it thus would have been obvious to one of ordinary skill in 
the art at the tinie of the invention to modify the teaching of Milne with the teaching of Jang to 
include "a pre-processor", in order to provide a multimedia system having a time base correction 
fimction between the incoming audio sample rate and that which is derived from the system 
master clock, as taught by Jang at col. 13, lines 38-41. 

Therefore, the claim language as presented is still read on by the Milne and Jang 
references at the cited paragraph in the claim rejections. Arguments as raised are moot since all 
claim limitations relevant to this issue have been addressed accordingly 
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Per (c), with regards to claim 13, Milne teaches the step of "an audio sample pre- 
processor configured to receive and preprocess audio samples before the samples are rendered by 
are rendered by a renderer that comprises part of a media player, " at col. 16, lines 62-64, col. 15, 
lines 39-59, and Fig. 25. It is brought to Applicant's attention that as shown in Fig. 25, the audio 
sample is provided frequency data (i.e. clock rate) associated with each sample before the 
samples are rendered. Note that the renderer corresponds to the amplifier (i.e. audio card, which 
is a part of the media player) rendering the audio sample after the host processor (Fig. 25). 

Milne teaches "the characterizing data comprising a time stamp associated with each 
audio sample, the time stamp being assigned in accordance with when the audio sample is 
calculated to be rendered by the renderer" at col. 19, lines 1-12. It is noted that the visual clock 
object 3720 is synchronized with the audio clock object 3720 as shown in Figure 37. 

Milne teaches "the muhiple data structures configured to hold the characterizing data, 
each data structure being associated with an audio sample" at col. 17, lines 13-58. The data 
structure corresponds to the "graphic objects related by time" (col. 17, lines 18), 

Milne teaches "an audio rendering object configured to call the audio sample to ascertain 
the characterizing data associated with an audio sample that is currently being rendered by the 
renderer" at col. 16, lines 21-39, that is, "The audio player can synchronize to and external clock. 
In this case, the audio player must insure that the sample being reproduced by the speaker 
corresponds to the current time of the external clock". It should be noted that the rendering 
object corresponds to an audio rendering object. 

Milne teaches "the audio sample pre-processor being configured to ascertain said 
characterizing data by querying the renderer for a time associated with the current-rendered 
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audio-sample, and then using said time to identify a data structure having a time stamp that is 
nearest in value to said time" at col. 16, lines 21-39, col. 19, Hnes 1-11, col. 17, lines 13-58. 
Note that the characterizing data corresponds to the current time (col, 16, line 32) which is 
examined to determine if it needs to speed up or slow down the play back rate or jump to a new 
position in the audio sequence in order to insure that the correct sample are being played from 
the speaker (col. 16 lines 33-36), and the data structure corresponds to the "graphic objects 
related by time" (col. 17, lines 18). 

Milne teaches "one or more effects configured to receive characterizing data that 
associated with the data structure having the timestamp that is nearest in value to said time, and 
use the characterizing data to render a visualization that is synchronized with the audio sample 
that is being rendered by the renderer" at col. 16, lines 22-39, col. 19, lines 1-11 and col. 17, 
lines 13-58. The data structure corresponds to the "graphic objects related by time" (col. 17, 
lines 18). 

Although the function of the Milne's processor is just as same as the function of the 
present invention's pre-processor which is provides the frequency data (Specification, page 19, 
lines 4-5), Milne does not use explicitly the term "pre-processor". However, Jang teaches the 
audio pre-processor 1 17 in Figure 6. Both Milne and Jang are direct to the same field as the 
synchronization between the audio and video data based on the audio sample rate, it thus would 
have been obvious to one of ordinary skill in the art at the time of the invention to modify the 
teaching of Milne with the teaching of Jang to include "a pre-processor", in order to provide a 
multimedia system having a time base correction function between the incoming audio sample 
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rate and that which is derived from the system master clock, as taught by Jang at col 13, lines 
38-41. 

Therefore, the claimed invention as represented in the claims does not represent a 
patentable over the art of record. 

Per (d), with regards to claim 23, Milne teaches the step of "receiving multiple audio 
samples" at col 15, line 61 to col 16, line 20. It is noted that, the "82,100^ sample in the audio 
sequence" corresponds to the multiple audio samples. 

Milne's reference teaches the step of "determining when an audio sample is being 
rendered by the media player renderer" at col 16 lines 21-39, that is, "the audio player must 
insure that the sample being reproduced by the speaker corresponds to the current time of the 
external clock". 

Milne teaches the step of "responsive to said determining, using the characterizing data 
that is associated with the audio sample that is being rendered to provide a visualization" at col 
16, lines 21-39, col 19, lines 1-1 1, col 17, lines 13-58. It should be noted that the characterizing 
data corresponds to the current time (col 16, line 32) which is examined to determine if it needs 
to speed up or slow down the play back rate or jump to a new position in the audio sequence in 
order to insure that the correct sample are being played from the speaker (col 16, lines 33-36), 
and the data structure corresponds to the "graphic objects related by time" (col. 17, Hnes 18). 

Milne teaches "pre-processing the audio samples before they are rendered by a media 
player renderer, the pre-processing providing characterizing data for each sample" at col 16, 
lines 62-64, col 15, lines 39-59, and Fig. 25. More specifically, as shown in Fig. 25, the audio 
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sample is provided characterizing data (i.e. clock rate) associated with each sample before the 
samples are rendered. It should be noted that the DAC and amplifier render the audio sample 
after the host processor (Fig. 25). 

It should be understood that the function of the Milne's processor is closely as same as 
the function of the Applicant's pre-processor which provides the frequency data. Milne, 
however, does not use explicitly the term "pre-processor". Jang teaches the audio pre-processor 
117 in Figure 6. Both Milne and Jang are direct to the same field as the synchronization between 
the audio and video data based on the audio sample rate, it thus would have been obvious to one 
of ordinary skill in the art at the time of the invention to modify the teaching of Milne with the 
teaching of Jang to include "a pre-processor" in order to provide a multimedia system having a 
time base correction function between the incoming audio sample rate and that which is derived 
from the system master clock, as taught by Jang at col. 13, lines 38-41. 

Arguments as raised are moot since all claim limitations relevant to this issue have been 
addressed accordingly 

Per (e), with regards to claim 31, Milne teaches "calling the media player renderer for a 
time associated with a currently-rendered audio sample" at col. 16, lines 21-39. It is noted that 
the clock object current time corresponds to "time", which associated with the 88,200^ sample in 
the audio sequence (see col. 16, lines 26-27), and the audio player corresponds to the media 
player. 

Milne's reference teaches "using the time to select a data structure containing 
characterizing data associated with the current-rendered audio sample" at col. 17, lines 13-58 and 
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Fig. 37. Note that the data structure corresponds to the "graphic objects related by time" (col. 
17, line 18). 

Milne teaches "providing the characterizing data to a component for rendering 
visualization" at col. 19, lines 1-9, Fig. 37. It is noted that Figure 37 shows the audio clock 
object 3720 being synchronized with the visual clock object 3710. 

Milne teaches "calling an audio sample pre-processor for characterizing data that is 
associated with an audio sample that is currently being rendered by a media player rendered" at 
col. 16, lines 21-39, that is, "The audio player can synchronize to an external clock. In this case, 
the audio player must insure that the sample being reproduced by the speaker corresponds to the 
current time of the external clock". It should be understood that the rendering object corresponds 
to an audio rendering object. 

The function of the Milne's processor is to provide the frequency data, which is as same 
as the function of the present inventions pre-processor (Specification, page 19, lines 4-5), but 
Milne does not explicitly disclose the term "pre-processor". However, Jang teaches the audio 
pre-processor 1 17 in Figure 6. Both Milne and Jang references are directed to the same field as 
the audio-to-video synchronization, the combination to employ the pre-processor providing 
frequency data associated with each sample as taught by Jang in the Milne's system, in order to 
correct the time-base the incoming audio sample rate based on the master clock as taught by Jang 
at col. 13 (32-42). 

Per (f), with regards to claim 21, Van Zoest teaches "a timestamp module for assigning 
timestamps to audio samples that are to be rendered by a media player renderer" at col. 4, lines 
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56-65, that is, "a time stamp, and session data". It is noted that the session data of Van Zoest 
could be audio file, video file, audio-video file (col. 5, lines 10). 

Van Zoest teaches "a spectaim analyzer for processing the audio samples to provide 
frequency data associated with the audio sample" at col. 16, lines 9-15, that is, "the verification 
process runs a Fast Fourier Transform ("FFT") algorithm in each WAV samples to generate their 
respective power spectrums". 

Van Zoest teaches "multiple data structures each of which being associated with an audio 
sample, the data structures each containing timestamp data and fi*equency data for associated 
audio sample" at col. 5, lines 8-19, that is, "The stored information may include the titles of 
works, names of the artist, information about the band or performance, UPC data, or any 
information about the works". 

Van Zoest teaches "the system being configured to use the timestamp data to ascertain a 
data structure associated with an audio sample that is currently being rendered by the media 
player renderer and provide the frequency data associated with that audio sample so that the 
frequency data can be used to render a visualization associated with that audio sample" at col. 8, 
lines 26-42. Note that the "any information" corresponds to the timestamp associated with the 
content database (i.e. audio file). 

Therefore, the Van Zoest system cannot be distinguished fi*om the claim invention since 
Van Zoest teaches all such elements as discussed. 
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Per (g), with regards to claim 29, Milne teaches "receiving muhiple audio samples" at 
col. 15, line 61 to col. 16, line 20. It is noted that "the 44,100 samples" corresponds to the 
multiple audio samples. 

Milne teaches "maintaining frequency data and a timestamp for each sample in a data 
structure" at col. 16, lines 21-59. It should be noted that the timestamp corresponds to the 
external clock's object current time, and the frequency data corresponds to the clock rate (col. 16, 
line 48). 

Milne teaches "determining when an audio sample is being rendered by the media player 
renderer" at col. 16, lines 21-39, that is, "the audio player must insure that the sample being 
reproduced by the speaker corresponds to the current time of the external clock". 

Milne teaches "ascertaining a time associated with a currently-rendered sample at col. 16, 
lines 21-39 as "the audio player must insure that the sample being reproduced by the speaker 
corresponds to the current time of the external clock". 

Milne teaches "selecting a data structure having a timestamp that is nearest the time" at 
col. 17, lines 13-58. Note that the data structure corresponds to the "graphic objects related by 
time" (col. 17, line 18). 

Milne teaches "providing frequency data associated with the selected data structure to a 
component configured to use the frequency data to render the visualization" at col. 17, lines 15- 
27, col. 19, lines 1-1 1, col. 16, lines 40-59, Fig. 37. The frequency data corresponds to the clock 
rate at col. 16, line 48. The using of this for visualization is shown in Figure 37. The selecting 
data structure corresponds to "A graphic sequence also provides a member function that maps a 
time within the duration of the sequence to a particular graphic object" (col. 17, lines 23-25). 
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Milne does not specifically teach a pre-processor and a Fast Fourier Transform. 

Although the function of the Milne's processor is as same as the function of the present 
invention's pre-processor which provides the frequency data (Specification, page 19, lines 4-5), 
Milne does not specifically use the term "pre-processor" for his component. Hov^ever, Jang 
teaches the audio pre-processor 1 1 7 in Figure 6. Both Milne and Jang direct to the same field as 
the synchronization between the audio and video data based on the audio sample rate, it thus 
would have been obvious to one of ordinary skill in the art at the time of the invention to modify 
the teaching of Milne with the teaching of Jang to have "a pre-processor" in order to provide a 
multimedia system having a time base correction function between the incoming audio sample 
rate and that which is derived from the system master clock, as taught by Jang at col. 13, lines 
38-41, 

And Zoest teaches the Fast Fourier Transform at col. 16, lines 9-15, that is, "the 
verification process runs a Fast Fourier Transform ("FFT") algorithm in each WAV samples to 
generate their respective power spectrum". 

Milne, Zang, Zoest are directed to the same field as accessing the mulitmeadia objects in 
the network, it would have been obvious to one of ordinary skill in the art at the time of the 
invention to employ the audio pre-processor of Zang, which comprises a Fast Fourier Transform 
in order to provide a multimedia system having a frequency analysis of data as taught by Zoest at 
col. 16 line 17. By doing this, the system could verify and identifying the corresponding 
portions of the samples with maximum correlation, as taught by Van Zoest at col. 16, lines 4-6. 
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Per (h), with regards to claim 34, Milne teaches "query for frequency data that is 
associated with an audio sample that is currently being rendered by the media player renderer" at 
col. 16, lines 40-59. The frequency data corresponds to the clock rate at (col 16, line 58). 

Milne's reference teaches the step of "query for a time associated with the currently- 
rendered audio sample" at col. 16, lines 21-39. It should be noted that the associated time 
corresponds to the external clock object's current time, and the currently rendered audio sample 
corresponds to the 88, 200^ samples in the audio sequence. 

Milne's reference teaches the use of time to select a data structure containing frequency 
data associated with the currently-rendered audio sample at col. 17, lines 13-27. It should be 
noted that the selecting step corresponds to the function mapping a time within the duration of 
the sequence to a particular graphic objects. 

Milne teaches " providing the frequency data to a component that uses the frequency data 
for rendering a visualization" at col. 17, lines 15-27, col. 19, lines 1-11, and col. 16, lines 40-59, 
Figure 37. It should be noted that the frequency data corresponds to the clock rate at col. 16, line 
48, and a visualization is shown in Figure 37, and the selecting data structure corresponds to "A 
graphic sequence also provides a member function that maps a time within the duration of the 
sequence to a particular graphic objects" (col. 17, lines 23-25). 

Milne teaches the pre-processing audio sample, which is performed by a host processor 
as shown in Fig. 25. But Milne does not specifically disclose the term "pre-processor". 

Milne also does not teach the using of a Fast Fourier Transform. 

However, Jang teaches the audio pre-processor 117 in Fig. 6, col. 13, lines 14-42. And 
Zoest teaches the Fast Fourier Transform at col. 16, lines 9-15, that is, "the verification process 
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mns a Fast Fourier Transform ("FFT") algorithm in each WAV samples to generate their 
respective power spectrum". 

Milne, Jang, Van Zoest are directed to the same field as accessing the multimedia objects 
in the network, the incorporation of Jang, Van Zoest in the combined system would have 
enhanced the performance of the system. By doing this, the system could verify and identifying 
the corresponding portions of the samples with maximum correlation, as taught by Van Zoest at 
col. 16, lines 4-6. 

Therefore, the claimed invention as represented in the claims does not represent a 
patentable over the art of record. 

Per (i), with regards to claim 35, Prasad teaches "defining a fi*ame rate at which 
visualization frames of a visualization are to be rendered" as "the pre-determined fi-ame rate 
maybe; for example, 15 frames-per-seconds" (col. 6, line 19-21). 

Prasad teaches "setting a threshold associated with an amount of time that is to be spent 
rendering a visualization frame" as "The synchronization information S may include any 
information that indicates how much of the audio data stream" (col, 6, lines 44-45). 
Prasad teaches the step of "monitoring the time associated with rendering individual 
visualization frames" at col. 6, lines 28-53. The monitoring corresponds to the step determining 
the total elapsed time in col. 6, line 5 1 . 

Prasad teaches "determining whether a visualization frame rendering time exceeds the 
threshold" at col. 6, lines 28-53, that is, "the synchronization information S may be an indication 
of the number of samples played, this indication can be used by the video playback process 37 to 
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determine the total elapsed time for which the audio stream 48 has been playing. Alternatively, 
the information S may specify the elapsed time explicitly"(col. 6, lines 48-53). 

Prasad teaches "providing an effective frame that is longer than the defined frame rate if 
the determined visualization frame rendering time exceeds the threshold" at col. 6, lines 28-59. 
The providing an effective frame corresponds to the step of adjusting the rate of playback of the 
recorded video stream 41 (col. 6, line 42). 

Prasad does not teach the characterizing data associated with the audio samples. 
However, Milne teaches "the visualization frames being rendered fi*om characterizing data that is 
computed from audio samples and which is used to create the visualization" at col. 17, lines 15- 
39 and Fig. 37. It should be noted that the data structure corresponds to the "graphic objects 
related by time" (col. 17, lines 18). Moreover, as shown in Fig.37 Milne shows the audio clock 
object 3720 being associated with the visual clock object 3710 (col 19, lines 8-1 1). 

Prasad and Milne are directed to the same field as the method and apparatus for 
synchronization audio and video stream. It thus would have been obvious to one of ordinary 
skill in the art at the time of the invention to employ the characterizing data computed from 
audio sample as taught by Milne to the system of Prasad in order to display the visual clock 
object being synchronized with the audio clock object as taught by Milne at col. 19, lines 8-11. 

Arguments as raised are moot since all claim limitations relevant to this issue have been 
addressed accordingly. 
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Per (j), with regards to claims 38, Prasad teaches "set a threshold associated with an 
amount of time that is to be spent rendering a visualization frame for a given frame rate" at col. 
6, lines 19-21, that is, "The pre-determined frame rate may be, for example, 15 rates-per- 
second". 

Prasad teaches "monitoring the time associated with rendering individual visualization 
frames" at col. 6, lines 28-53. The monitoring corresponds to the step determining the total 
elapsed time (col. 6, Hne 51). 

Prasad teaches "determining whether a visualization frame rendering time exceeds the 
threshold" at col. 6, lines 28-53, that is, "the synchronization information S may be an indication 
of the number of samples played, this indication can be used by the video playback process 37 to 
determine the total elapsed time for which the audio stream 48 has been playing. Alternatively, 
the information S may specify the elapsed time explicitly" (col. 6, lines 48-53). 

Prasad teaches "providing an effective frame that is longer than the defined frame rate if 
the determined visualization frame rendering time exceeds the threshold" at col. 6 lines 28-59. 
The providing an effective frame corresponds to the step of adjusting the rate of playback of the 
recorded video stream 41 (col. 6, line 42). 

Prasad does not teach the characterizing data associated with the audio samples. 
However, Milne teaches "the visualization frames being rendered from characterizing data that is 
computed from audio samples and which is used to create the visualization" at col. 19, lines 1-9, 
col. 17, lines 15-39, Fig. 37. Note that as shown in Fig.37, Milne discloses the audio clock 
object 3720 being associated with the visual clock object 3710 (col. 19, lines 8-11). 
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Prasad and Milne are directed to the same field as the method and apparatus for 
synchronization audio and video stream. Therefore, it would have been obvious to one of 
ordinary skill in the art at the time of the invention v^as made to employ the characterizing data 
computed from audio sample as taught by Milne to the system of Prasad in order to display the 
visual clock object being synchronized with the audio clock object as taught by Milne at col. 19, 
lines 8-11. 

Accordingly, the claimed invention as represented in the claims does not represent a 
patentable over the art of record. 

Conclusion 

14. Applicant's amendment necessitated the new ground(s) of rejection presented in 
this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). 
Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THDREE 
MONTHS fi-om the mailing date of this action. In the event a first reply is filed within TWO 
MONTHS of the mailing date of this final action and the advisory action is not mailed until after 
the end of the THREE-MONTH shortened statutory period, then the shortened statutory period 
will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 
CFR 1. 136(a) will be calculated from the mailing date of the advisory action. In no event, 
however, will the statutory period for reply expire later than SIX MONTHS from the date of this 
final action. 
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The prior art made of record and not relied upon is considered pertinent to applicant's 
disclosure. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Miranda Le whose telephone number is (703) 305-3203. The 
examiner can normally be reached on Monday through Friday from 8:30 AM to 5:00 PM. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, John E. Breene, can be reached on (703) 305-9790. The fax number to this Art Unit 
is (703) 872-9306. 

Any inquiry of a general nature or relating to the status of this application should be 
directed to the Group receptionist whose telephone number is (703) 305-3900. 

Miranda Le 
February 12, 2004 



